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Key Takeaways 


In partnership with the Louisiana Department of Education (LDOE), and through a grant 
from the Walton Family Foundation, Mathematica has developed measures of each 
Louisiana public high school’s promotion power—the school’s impact on the long-term 
success of its students, as indicated by high school graduation, college or career 
readiness, college enrollment and persistence, and success in the job market. These new 
measures will enable Louisiana to become one of the first states to report on high 
schools’ success in improving the prospects for their students in higher education and 
the workforce. 


Measures of promotion power aim to fairly compare schools serving different 
populations of students. The measures are based on statistical models that identify 
schools’ contributions to students’ long-term outcomes separately from other factors, 
such as prior achievement and demographic characteristics (Figure ES.1). The statistical 
models are designed to create (to the extent possible) a level playing field that permits 
fair comparisons of schools that serve different student populations. 


Figure ES.1. Promotion power measures separate schools’ contributions 
to students’ long-term outcomes from other factors 
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Measures of promotion power substantially reduce or even eliminate the 
relationship between student poverty and the school’s measure of performance, 
thereby assessing school effectiveness more accurately. Student background 
characteristics and preparation before high school affect student success, so schools 
serving advantaged students tend to have better outcomes regardless of how well they 
are serving those students. For example, Louisiana high schools with low poverty 
rates—as measured by the percentage of students eligible for free or reduced-price 
lunches—tend to have higher college enrollment (Figure ES.2, left panel). In contrast, 
schools can do just as well in promotion power regardless of the economic advantages or 
disadvantages of students they serve (Figure ES.2, right panel). 
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This report includes details about 
the outcomes, data, and 
statistical model used to 
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Louisiana public high schools. 
The measures of student success 
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DATA 


Student background and high 
school graduation data are from 
LDOE administrative records. 
College enrollment and 
persistence data come from the 
National Student Clearinghouse. 
Earnings data are from the 
Louisiana Workforce 
Commission. Section II.A 
contains details about data 
security procedures. 


METHODS 


To measure schools’ promotion 
power, we used a statistical 
model to account for student 
test scores at the end of 8th 
grade as well as student 
demographic characteristics. 
Each measure of promotion 
power is based on data from two 
cohorts of students. 
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Figure ES.2. Schools with higher poverty rates have lower rates of college enrollment, but promotion 
power puts schools on a level playing field 
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Notes: Each dot represents a Louisiana high school. The green line indicates the relationship between the proportion of students 
eligible for free or reduced-price lunch and each school measure. 


Louisiana high schools vary widely in their success in promoting high school graduation, college enrollment, and 
eventual earnings. The promotion power measures distinguish 41 to 67 percent of Louisiana high schools from average across 
outcomes. A typical student who moved from an average high school to one at the 95th percentile of each promotion power 
distribution could expect substantially better outcomes (Figure ES.3). 


Figure ES.3. Expected outcomes of students attending high schools with high promotion power are 
substantially better than those of similar students attending high schools with average promotion power 
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Notes: Each set of bars depicted in this figure is based on a separate promotion power distribution and represents different 
schools. It is unlikely that a single school would be at the average or 95th percentile of all promotion power models. 


Schools with a positive impact on one measure of promotion power are more likely to positively impact other student 
long-term outcomes, but many schools show varying effectiveness for different outcomes. Table ES.1 reports the 
correlations between measures of promotion power for each school across the outcomes included in our analysis. The 
correlations are positive and statistically significant across the majority of outcomes. For example, schools that successfully 
promote high school graduation also tend to promote college enrollment and earnings. Even so, some of the correlations are not 
very large: high schools that are particularly good at promoting college enrollment and persistence, for example, do not 
necessarily promote strong earnings for their students at age 26. This highlights the need to include multiple measures of 
promotion power to capture the different ways schools can influence student long-term outcomes. 


Table ES.1. Promotion power impacts are positively correlated across outcomes 


Credential College College Earnings 
Outcome completion enrollment persistence (age 26) 
High school graduation 0.36 0.57 0.15 0.27 
Credential completion -- 0.24 n/a n/a 
College enrollment a -- 0.72 -0.07* 
College persistence -- -- -- -0.03* 


Notes: The correlations in this table are based on the most recent available cohorts for each pair of outcomes. The correlations 
between credential completion, college persistence, and earnings are not reported because we do not have data for the 
same set of cohorts for those outcomes. All correlations are significantly different from zero at the 0.05 level, except those 
indicated by the + symbol. 


n/a = not applicable. 
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I. INTRODUCTION 


The Louisiana Department of Education (LDOE) seeks to measure the promotion power of its high 


schools—the effectiveness of each high school in promoting the long-term success of its students— 
as indicated by high school graduation, completion of a college or career readiness credential, 
college enrollment and persistence, and ultimately earnings. 


Several studies have shown that some high schools can have significant positive impacts on students’ 
long-term outcomes. For example, some studies have found large effects of Catholic schools on high 
school and college graduation and future earnings (Neal 1997; Grogger and Neal 2000; Evans and 
Schwab 1995). Sass et al. (2016) and Booker et al. (2011) found significant impacts of charter high 
schools in Chicago and Florida on graduation, college attendance and persistence, and earnings in 
adulthood. Angrist et al. (2016) showed that Boston charter schools have positive impacts on 
postsecondary preparedness and the selectivity of colleges their students attend. 


Though research has shown high schools can substantially influence students’ long-term outcomes, 
few states report information that could be used to measure the effectiveness of high schools at 
improving these outcomes. Although some states report the college enrollment rate of high school 
graduates, almost no states currently report the impacts each high school has on college enrollment or 
other longer-term outcomes such as college persistence, and earnings (Achieve 2017).! The 
distinction between reporting the raw outcomes and the schools’ impacts on those outcomes is 
critical, because the raw outcomes are affected by many factors that are outside the control of 
schools, such as the socioeconomic status and education levels of students’ parents (Aldeman 2015). 
A simple comparison of the college enrollment rates of students from a high-income suburban high 
school and students from a low-income rural high school, for example, cannot tell us whether one 
school is doing better at promoting college enrollment for the students it serves. 


With the adoption of promotion power measures, Louisiana will become one of the first states to 
measure and report how well each public high school is advancing its students’ long-term success— 
levelling the playing field for high schools, regardless of the advantages and disadvantages of the 
students they serve. This will enable the state to identify high-performing high schools that can serve 
as exemplars of how to improve long-term outcomes for students, as well as low-performing high 
schools that might need help. 


The promotion power measures described in this report are based on statistical models that account 
for student 8th-grade test scores and other background characteristics such as poverty status, 
disability, and 8th-grade attendance and suspensions. They are similar to value-added models of 
student test score growth commonly calculated for teachers and schools. Researchers have shown that 
teacher value-added models produce valid and reliable measures of teachers’ impacts on growth in 
student test scores (Kane and Staiger 2008; Kane et al. 2013; Chetty et al. 2014a) and that these 


' Kansas is the only state we are aware of that currently reports on a measure of high school student college enrollment and persistence that 
accounts for student background characteristics. More details about the measure Kansas uses is available at https://kasb.org/blog/ksde-measures- 
of-postsecondary/. 
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measures are related to student college and earnings outcomes (Chetty et al. 2014b). Similarly, 
researchers evaluating school value-added measures have found that they contain minimal bias 
(Deutsch 2012; Deming 2014), though the measures can be improved upon if data from school choice 
lotteries are available (Angrist et al. 2017). 


We cannot be certain that the statistical adjustments in promotion power models succeed in making 
the playing field 100-percent level across high schools, because school choice lotteries have not yet 
been used to examine the causal validity of promotion power models. Even so, making statistical 
adjustments for important student characteristics (such as 8th-grade achievement) is likely to 
substantially improve on raw measures of student outcomes in assessing high schools’ effectiveness. 


Calculating promotion power measures can be thought of as a two-step process. In the first step, we 
use a Statistical model to estimate the likelihood that each student would achieve a certain outcome, 
such as college enrollment, based on that student’s background characteristics. In the second step, we 
compare the actual college enrollment rates of each high school to the expected rates based on the 
statistical model. The promotion power measure for college enrollment is the actual deviation above 
or below the expected college enrollment rate. The measure answers the following question: To what 
extent does the actual college enrollment rate of a high school’s students exceed (or fall short of) the 
level that students with similar 8th-grade test scores and background characteristics would reach if 
they were taught by the average school in the state? 


The next chapter provides details about the data sources for the outcomes included in the promotion 
power measures as well as the student background characteristics accounted for in the statistical 
models. Chapter III provides the technical details of the specification of the statistical models, and 
Chapter IV summarizes the resulting promotion power measures. 


PROMOTION POWER TECHNICAL REPORT MATHEMATICA 


II. OUTCOMES AND BACKGROUND CHARACTERISTICS 


USED IN THE PROMOTION POWER MODELS 


A. Outcomes analyzed in the promotion power models 


This section briefly describes the outcomes used to measure high school promotion power. Appendix 
Section A contains additional details about the construction of each measure. Data from the 
Louisiana Workforce Commission (LWC) were handled in accordance with the measures outlined in 
the agreement between LWC and LDOE. Strict security measures are maintained for LWC data, such 
that only a small number of LDOE staff who have signed nondisclosure agreements can access them. 
Additionally, data from the National Student Clearinghouse (NSC) were handled in accordance with 
the measures outlined in the agreement between LDOE and NSC. 


1. High school graduation 


The first promotion power measure we analyzed is the impact schools have on the likelihood that 
students will graduate from high school on time. On-time high school graduation occurs when a 
student graduates within four years of the first time that student enrolled in a Louisiana public high 
school in 9th grade. If a student repeats 9th grade and does not catch up to the rest of his or her 
cohort within the next three years, that student is counted as not graduating high school on time. The 
two most recent cohorts of high school graduates with data available for our analysis graduated in 
spring 2016 and spring 2017. 


Outcome: High school graduation 


Definition: Graduating high school on time — within four years of the first time a student entered 9th 


grade 
Data source: LDOE administrative data from the 2015-16 and 2016-17 school years 


2. Completion of a college or career readiness credential 


The next outcome used for promotion power measures whether students have completed a college or 
career readiness credential (or “credential completion” for short) by the time students graduate high 
school. Louisiana defines credential completion as either (1) completing college-level coursework by 
taking an Advanced Placement class, an International Baccalaureate course class, or a Dual 
Enrollment college class; or (2) demonstrating proficiency with an industry-valued skill set that is 
recognized by the Workforce Investment Council. Similar to the high school graduation outcome, the 
credential completion analysis includes students graduating in spring 2016 or spring 2017. 
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Outcome: Completion of a college or career readiness credential 


Definition: Completing college-level coursework or demonstrating proficiency with an industry-valued 


skill set 
Data source: LDOE administrative data from the 2015-16 and 2016-17 school years 


3. College enrollment 


We also use as an outcome whether students enroll in college in the fall following their expected high 
school graduation date. Enrollment in either a two-year or four-year college counts toward this 
measure. Data are from the NSC and based on college enrollment during fall of 2016 and the fall of 
2017. 


Outcome: College enrollment 


Definition: Enrolling in a two-year or four-year college during the fall semester after a student’s 


expected high school graduation date 
Data source: NSC data from the 2016-17 and 2017-18 school years 


4. Multi-year college persistence 


Our ideal measure of college attainment for promotion calculations would be earning a bachelor’s 
degree from a four-year college. Though the NSC tracks college graduation outcomes for students, 
institutions sometimes under-report degree completion (Dynarski et al. 2015). This issue is especially 
prevalent among Louisiana colleges in the NSC data; analyses by LDOE staff show that college 
graduation can be under-reported—or not reported at all—by certain colleges in some years. When 
we tested using NSC data on college graduation in promotion power models, the results tended to be 
statistically noisy and unreliable. We therefore recommend against including a measure of college 
graduation based on NSC data in the promotion power models. However, if LDOE were able to 
obtain graduation data from in-state public colleges from the Louisiana Board of Regents, that might 
be sufficient to supplement the NSC data and include college graduation as an outcome in future 


years. 


Whereas graduation is poorly measured in the NSC data, several possible measures of persistence in 
college are well reported. To select one of these possible persistence measures, we examined the 
enrollment patterns of students who are reported as graduating in the NSC data (Appendix Section 
A). This analysis led us to the following measure of multi-year college persistence (referred to as 
“college persistence” for short): attending college for at least four years—at least two of which are at 
a four-year college—within five years of a student’s expected high school graduation date. A small 
percentage of students did not satisfy these criteria but were classified as completing a four-year 
degree according to the NSC data; these students also counted toward this measure for calculating 
promotion power for college persistence. 
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The college persistence outcome uses earlier cohorts of Louisiana students than the college 
enrollment outcome, such that data from the NSC on the 2016-17 or 2017-18 school years would 
represent the fifth year after the cohorts were expected to graduate from high school. That is, the 
college enrollment outcomes use cohorts graduating high school in spring 2016 and spring 2017, 
whereas the college persistence outcome uses cohorts graduating in spring 2012 and spring 2013. 


Outcome: Multi-year college persistence 


Definition: Attending college for at least four years—at least two of which are at a four-year college— 


within five years of a student’s expected high school graduation date 
Data source: NSC data from the 2012-13 through 2017-18 school years 


5. Earnings at age 26 


The longest-term promotion power outcome measure we examined is earnings in Louisiana at age 
26.2 We chose age 26 for our analysis because it is the latest age for which data on Louisiana 
students are available, and it provides enough time for students who attended college to complete 
their undergraduate degree and start working. 


The measure captures the amount a student earned during the calendar year eight years after the 
student’s expected high school graduation date (the calendar year that students in this cohort turn 
26), through an employer that reports earnings to Louisiana’s unemployment insurance system. 
Louisiana students who end up working out of state, join the military, or are self-employed do not 
appear in the unemployment insurance data and are therefore classified as having zero earnings 
according to this measure. 


Because students who attend college out of state might be more likely to work out of state after 
college, we excluded from the analysis the 3 percent of students who attend an out-of-state college 
and do not appear as employed in Louisiana in any available year of earnings data. We also compared 
this main measure of promotion power to measures based on two alternative approaches: 

(1) including these students in the analysis and counting them as having zero earnings, and 

(2) excluding all students who never appear as employed in Louisiana in any available year— 
representing 24 percent of students. The results of these comparisons are described in Appendix 
Section D. 


Data for this analysis are based on quarterly earnings records from the LWC for calendar years 2014 
and 2015, which are the most recent years of data LDOE staff currently have access to. The annual 
earnings measure is a sum of earnings in the four quarters of each calendar year. To limit the 


influence of outlier earnings observations, we top-coded annual earnings at $105,000.3 Less than 1 


? We also examined an indicator for being employed (having non-zero earnings) at age 26, as a possible additional promotion power outcome. 
However, we found that this outcome did not vary substantially across schools and that the promotion power measure based on it was not likely 
to provide meaningful information about school effectiveness. 


3 Top-coding is a commonly used approach to handle outlier earnings observations (for example, Couch and Placzek 2010; Chetty et al. 2014b). 
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percent of annual earnings observations included in our analysis were initially greater than $105,000 
and were top-coded. 


Outcome: Earnings at age 26 


Definition: Annual earnings reported to Louisiana’s unemployment insurance system during the 
calendar year that students in this cohort turn 26 


Data source: Quarterly earnings data from the LWC from 2014 and 2015 


6. Outcome summary statistics 


Table II.1 displays averages and standard deviations for each outcome used in the promotion power 
models. Approximately three-quarters of students graduate high school on time, almost half enroll in 
college, and a little less than one-quarter persist for multiple years in college. The average earnings 
amount might appear low because unemployed individuals are counted as having $0 earnings in this 
calculation. Among individuals with positive earnings, the average annual amount is $26,114. All of 
these measures show substantial variation among high schools across the state. 


Table II.1. Outcome summary statistics 


Across-school 


Average standard deviation Number of 

(percentage or (percentage points Number of high 
Variable dollars) or dollars) students schools 
High school graduation 75.0 30.4 89,351 352 
Credential completion 44.7 24.7 89,351 352 
College enrollment 46.2 21.8 89,351 352 
College persistence 23.3 18.5 84,561 375 
Annual earnings at age 26 $17,863 $5,412 81,778 377 


Notes: Outcome summary statistics are based on the two most recent years of available data, which are 2015- 
16 and 2016-17 for the high school graduation outcome, 2016-17 and 2017-18 for college outcomes, 
and 2014 and 2015 for earnings. All students who enrolled in a Louisiana high school in 9th grade and 
have non-missing data on the necessary outcomes and background characteristics are included in the 
sample. The number of students and schools in the college persistence row is different from that in the 
college enrollment row, because it is based on earlier cohorts of students. The high school graduation, 
credential completion, and college enrollment rates will not exactly match those reported on LDOE’s 
website due to differences in which cohorts are included in the calculations and how the measures are 
constructed. Appendix Section A contains additional details. 
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B. Background characteristics used as control variables in the promotion 
power models 


The promotion power models include as control variables 8th-grade test scores and multiple other 
background characteristics for students. These account for the fact that high schools serve different 
populations of students who might be more or less likely to succeed on the outcomes described in the 
previous section, regardless of which high school they attend. 


The background characteristics are listed in Table II.2, along with a brief description of each 
variable. All characteristics come from LDOE administrative data. Appendix Section B contains 
summary Statistics and additional details about the construction of each measure. Appendix Section 
G reports the coefficients on each background characteristic variable in the promotion power models. 


Many value-added models of teacher and school effectiveness include race/ethnicity and gender as 
control variables (for example, Gonzalez et al. 2016; Chetty et al. 2014a; Walsh et al. 2014), while 
others exclude these variables (for example, Florida Department of Education 2015; Resch and 
Deutsch 2015; Isenberg and Walsh 2014). LDOE requested that we not use indicators for these 
characteristics in the promotion power models, for consistency with LDOE’s school and teacher 
value-added models. 


In Appendix Section D, we report results from a robustness check where we added indicators for 
race/ethnicity to the model. The correlations with the main results ranged from 0.91 to 0.98, 
depending on the outcome. The correlations were lowest for the college enrollment and persistence 
outcomes, indicating a stronger relationship between race/ethnicity and college outcomes than for 
high school graduation, after accounting for the other control variables included in the models. In 
another robustness check in which we added an indicator for gender, all of the correlations were 0.99 
and above. 


Table II.2. Background characteristics included in promotion power models 


Variable Description 

8th-grade test scores Louisiana Educational Assessment Program (LEAP) scores in English language arts, 
math, science, and social studies taken during the spring of each student’s 8th- 
grade year 

8th-grade absences The number of days a student was absent from school in 8th grade. We create an 


annualized measure of absences for students who were present in the district for 
less than the full year (see Appendix Section B for details). 


8th-grade suspensions The number of days a student received an in-school or out-of-school suspension in 
8th grade. We create an annualized measure of suspensions for students who were 
present in the district for less than the full year (see Appendix Section B for details). 


Attended 8th grade Because the number of absences and suspensions rates for students who attended 

fewer than 45 days 8th grade for fewer than 45 days are based on a small sample of days and likely to 
be unreliable, we treated the 8th-grade suspension and absence rates for these 
students as missing data and used this indicator to account for them. 


Over-age for grade An indicator for whether the student is two or more years over-age (16 or older) as 
of October 1st of their 9th-grade year 


Transfer high schools An indicator for whether the student transferred schools at some point during the 
student’s high school career 
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Variable Description 


Gifted status Indicator for whether the student was classified as gifted and talented 


Free lunch receipt Indicator for whether the student was eligible to receive free lunches 


Note: All of the characteristics described in this table use data from 8th grade, except for the indicator for 
students who transfer high schools and the over-age indicator (see Appendix Section B for details). 


© 
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III. TECHNICAL DETAILS FOR THE PROMOTION POWER 


MODEL SPECIFICATIONS 


A. General description of promotion power models 


The promotion power models are designed to measure the effectiveness of each high school in 
helping its students achieve the outcome of interest, after accounting for student background 
characteristics. The models generate a relative measure for each school that quantifies the difference 
between its students’ outcomes and the expected outcomes if those students attended the average 
high school in Louisiana. By controlling for student characteristics, the models seek to distinguish 
the effects of high schools from other factors influencing student outcomes that are outside the 
control of schools. 


Calculating the promotion power measure can be thought of as a two-step process. In the first step, 
we use a Statistical model to estimate the likelihood that each student would achieve a certain 
outcome, such as college enrollment, based on that student’s background characteristics. In the 
second step, we compare the actual college enrollment rates of each high school to the expected rates 
based on the statistical model. The promotion power measure for college enrollment is the deviation 
above or below the expected college enrollment rate. The measure answers the following question: 
To what extent does the actual college enrollment rate of a high school’s students exceed (or fall 
short of) the level that students with similar 8th-grade test scores and background characteristics 
would reach if they were taught by the average school in the state? 


To implement the calculation, we estimated a regression model that performs both of these steps 
simultaneously. We then accounted for measurement error in the resulting promotion power measures 
by adjusting schools’ estimates toward the average, depending on the precision of the estimate for 
each school. Finally, we averaged school estimates over two consecutive cohorts. The rest of this 
chapter discusses each stage of the estimation process in detail. 


B. Equations used for promotion power calculations 


1. Description of the regression model 


For each outcome, we estimated separate regressions for each cohort of students. Students are 
assigned to a cohort based on the first year they attended 9th grade. That is, if a student began 9th 
grade in fall 2014, and then was still in 9th grade in fall 2015 (because the student did not have 
enough credits to proceed to 10th grade), the student would be assigned to the 2014 cohort for the 
purpose of the promotion power models. As such, each student is assigned to only one cohort. The 
regression models take the following form: 


(1) Vie = Bire + Xit Ve + Die Ot + Cit 


where, Yj; is the outcome variable for student /in cohort & By represents a vector of students’ 
baseline 8th-grade Louisiana Educational Assessment Program (LEAP) scores. X;; is the set of other 
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student background characteristics listed in Table I.2; and D;, is the set of variables, one for each 
school, equal to 1 if student / attended that school, and equal to 0 otherwise. 6; is a vector containing, 
for each school, the estimated effect of attending that school. The ¢ subscripts indicate that the model 
is estimated separately for each cohort. 


This regression model is analogous to the fixed effects model that has been used widely in the 
literature to estimate effectiveness of schools and teachers. In the context of teacher value-added 
models, Guarino et al. (2015)—-~who refer to this model as “dynamic ordinary least squares”—have 
found that this model is robust to a variety of assumptions. 


2. Linear probability model used to calculate most promotion power measures 


All of the outcomes we analyzed except for earnings are binary: they only take the values of 0 or 1. 
Some researchers use non-linear regression models, such as logistic or probit models, for binary 
outcomes. We used the linear probability model, which essentially treats the binary outcome as if it 


were continuous, for several reasons: 


a. The coefficients from linear probability models are easy to interpret. The coefficient for a given 
school from the high school graduation model, for example, is the effect of attending that high 
school on the probability of graduating high school, in percentage points. Logistic models, on 
the other hand, generate coefficients in the form of log odds ratios, which are more challenging 
to interpret. 


b. The linear probability model can estimate effects for all school-cohort combinations, including 
those in which all students have the same value for an outcome. Logistic models cannot estimate 
an effect for schools in which, for example, all of the students or none of the students graduate 
high school. Because many schools have relatively small cohorts of students, there are a non- 
trivial number of schools in which all students have the same value for the outcome.‘ 


c. The linear probability model has generally been found to have good properties and to produce 
similar results to non-linear models in many settings (Wooldridge 2013). 


As a robustness check, we estimated the promotion power models via logistic regression and 
compared the results to those from the main model. Because the logistic regression drops schools and 
their students when it cannot estimate an effect for them (see Part b above), we first limited the 
estimation sample for the main model to only those students and schools used in the logistic 
regression for the purpose of this comparison. The correlation between the main results and the 
results from the logistic regression ranged from 0.85 to 0.98 across outcomes (see Table A.2 in 
Appendix Section D). 


4 Though not as prevalent as with a logistic model, there are some schools where all students have the same outcome about which the linear 
probability model does not provide useful information. See Appendix Section C for additional details. 


10 


PROMOTION POWER TECHNICAL REPORT MATHEMATICA 


3. Data on students who attend multiple high schools are weighted according to 
the time spent at each school 


To account for many students attending more than one school in their high school careers, we created 
distinct observations for each student-school combination and used weighted least squares so that 
each student contributes the same total weight. That is, a student who spent one-quarter of his or her 
high school career at school A and three-quarters at school B would contribute two records to the 
data: the first would be assigned a weight of 0.25 student equivalents toward school A’s total and the 
latter a weight of 0.75 student equivalents toward school B’s total.° This technique, referred to as the 
Full-Roster Method, was introduced by Hock and Isenberg (2017) as a way to account for students 
who are taught by multiple teachers in the same year and subject in value-added models. It has also 
been used to account for students who attend multiple schools in school value-added models (Resch 
and Deutsch 2015; Gonzalez et al. 2019; Walsh et al. 2019). As recommended by Hock and Isenberg 
(2017), we used cluster-robust standard errors to account for the correlation between multiple 
observations of the same student. 


To avoid generating imprecise estimates based on small samples of students, we required each school 
to have at least 10 student equivalents in the cohort to be included in the model. To avoid dropping 
students at schools that do not meet this requirement from the analysis, we reassigned the weight for 
these students to a single outside school record. Less than | percent of student-equivalent weight is 
assigned to the outside school record for each outcome. 


Students who do not spend 9th grade in a Louisiana public high school and transfer in at a later grade 
are excluded from the promotion power calculations. The majority of these students are missing 
information on 8th-grade test scores, meaning that we do not have sufficient information about their 
background before they enter high school to fairly compare them to other students in the analysis. 
Some students might spend 8th grade in Louisiana, transfer out of Louisiana public schools in 9th 
grade, and return at a later grade before finishing high school. These students are also excluded from 
the analysis, because they did not spend 9th grade in a Louisiana high school and were therefore not 
taught by Louisiana public schools for a duration comparable to other students. 


4. Two cohorts of students are used for each measure to increase precision 


The precision of promotion power estimates increases as the number of students included in the 
calculations for each school increases. To gain greater precision for the promotion power estimates, 
LDOE requested that we combine two cohorts when calculating each school’s promotion power 
measures. Table A.4 in Appendix Section E shows how the measures become more precise as 
multiple cohorts of students are combined. Precision increases substantially when two cohorts are 
used instead of one, but less so when the number of cohorts is increased to three, which helped 
inform LDOE’s decision to use two cohorts. 


One downside to combining estimates in this way is that the information used for promotion power 
measures becomes less current as the data go back one year further. However, given that promotion 


5 See Appendix Section C for further details. 
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power measures are already long-term measures of outcomes occurring years after students began 
attending high school, the increase in precision might outweigh the drawback of the data being less 
current by one year. 


To combine the estimates, we generated a weighted average of each school’s estimates for the two 
cohorts, in proportion to the number of student equivalents in each cohort for that school. That is, if a 
school has 50 student equivalents in one cohort and 100 in another, the first would receive a weight 
of 1/3, while the latter would receive a weight of 2/3. When calculating the standard error of the 
combined estimate, we again weighted by student equivalents and treated the two estimates as 
independent because they are based on different sets of students in each cohort. As with all averages 
of independent estimates, the standard error of the combined (averaged) estimate is lower than the 
standard errors of each one separately. 


5. Empirical Bayes shrinkage is applied to enhance the stability of the measures 


Even after restricting the analysis to schools with at least 10 student equivalents, it would still be the 
case that, by random chance, smaller schools would be more likely to have promotion power 
estimates far from the mean than larger schools. To address this issue, we employed empirical Bayes 
shrinkage following the procedure described by Morris (1983). This procedure moves school 
estimates toward the mean in proportion to their precision. That is, a school with fewer students, and 
thus less precision, will be “shrunk” toward the mean more than a school with more students. The 
theory behind this approach is that each school is assumed to be average until proven otherwise; the 
more precise the information is that the school is different from average, the less weight we place on 
the assumption that the school is average. An empirical Bayes shrinkage adjustment is commonly 
used in value-added models (for example, Angrist et al. 2017; Chetty et al. 2014a; Kane and Staiger 
2008). Appendix Section C provides additional details. 


6. Students with some missing background characteristic data are included in the 
model 


Students’ 8th-grade LEAP scores are important control variables in our model, as they measure 
students’ academic achievement just before entering high school. As such, we excluded from the 
model students who have no 8th-grade test score records, either because they did not attend a 
Louisiana school in 8th grade or for some other reason. Less than 5 percent of students had at least 
one of the four test scores from 8th grade (math, English language arts [ELA], social studies, or 
science), but were missing one or more of the other scores. To accommodate these students, we 
included indicators, one for each subject, for students who were missing an 8th-grade test score, and 
set scores for these students to the state average. 


For students who were enrolled in a Louisiana school for fewer than 45 days in 8th grade, we treated 
their annualized 8th-grade absence and suspension days as being missing, so as not to use noisy 
extrapolations of what their rates would have been in a full year of school. Similar to the approach 
for missing LEAP scores described above, we set the 8th-grade attendance and suspension days for 
these students to a constant and included an indicator for being enrolled fewer than 45 days in 8th 
grade. 
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To confirm that the promotion power measures are not sensitive to these modeling decisions, we 
estimated models that exclude any student who is missing any 8th-grade test score or was enrolled 
for fewer than 45 days in 8th grade. For each outcome, the correlation between the promotion power 
measures from the main model and the alternative model was greater than 0.99 (see Table A.2 in 
Appendix Section D). 


7. Limitations 


Two key limitations are important to note regarding the promotion power measures. One is that by 
examining high school effects on long-term outcomes, the time of outcome measurement is 
necessarily far removed from the time that students were in high school. Even though high school 
graduation is measured at the end of high school, and college enrollment shortly thereafter, the 
school factors that contribute to these outcomes begin as early as 9th grade. For example, a student 
who does not graduate on time may have started to fall behind in his or her courses and disengage 
from school in 9th grade, and would not contribute to his or her school’s promotion power measure 
until four years later. This presents a limitation in that the promotion power measures reflect school 
practices from years earlier. Because schools might improve or decline in effectiveness over time, a 


school’s most recent reported promotion power measure might not reflect its current effectiveness. 


Second, although the literature has generally found school value-added measures to be accurate, no 
studies have examined the accuracy and validity of promotion power measures on long-term 
outcomes. There could be reasons that value-added measures estimating growth in a single year on 
test score outcomes could be accurate while promotion power measures on long-term outcomes might 
be less so. For one, prior-year test scores could be more effective control variables for current-year 
test scores than 8th-grade test scores are for long-term outcomes. More generally, factors outside of 
the school’s control might exist, such as the level of parental involvement, that influence long-term 
outcomes but we do not capture in our data. These factors might be correlated with which school a 
student attends and could play a more prominent role in influencing long-term outcomes than they do 
for current-year test scores. 


Even though school promotion power measures are imperfect, they account for key characteristics 
outside schools’ control and thus can do much better than raw outcome measures at identifying 
schools’ impacts on those outcomes. 
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IV. RESULTS OF PROMOTION POWER MODELS 


We present a number of ways of examining the promotion power results. First, we report the extent 


to which promotion power is detecting meaningful differences across schools. We look at the 
difference between a school at the 95th percentile in the promotion power distribution for a given 
outcome relative to the average high school in Louisiana. This value represents the degree to which a 
typical student’s probability of attaining the outcome (graduating high school, enrolling in college, 
and so forth) would increase if the student moved from an average school to a school at the 95th 
percentile in promotion power for that outcome (Figure IV.1). Note that each set of bars depicted in 
Figure IV.1 is based on a separate promotion power distribution and represents different schools. It is 
unlikely that a single school would be at the average or 95th percentile of all promotion power 
models. 


For the latest cohorts available for each outcome, we found that students moving from an average to 
a 95th percentile school for each outcome would have their likelihood of (1) graduating from high 
school increase by 14 percentage points, (2) completing a college or career readiness credential 
increase by 24 percentage points, (3) enrolling in college increase by 13 percentage points, and 

(4) persisting in college increase by 12 percentage points. The earnings at age 26 of a student moving 
from the average school to the 95th percentile school for earnings promotion power would increase 
by $5,698. 


Figure IV.1. Expected outcomes of students attending high schools with high promotion 
power are substantially better than those of similar students attending high schools with 
average promotion power 
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Notes: Each set of bars depicted in this figure is based on a separate promotion power distribution and 
represents different schools. It is unlikely that a single school would be at the average or 95th percentile 
of all promotion power models. 


The difference in impact between the 95th percentile and average Louisiana high school on the 
promotion power measures is large, and within the range of impact estimates found in other studies. 
Catholic high schools have been found to increase high school graduation rates for urban minorities 
by 16 to 18 percentage points (Neal 1997; Grogger and Neal 2000). Another study found that the 
offer of a DC Opportunity Scholarship increased student high school graduation rates by 12 
percentage points (Wolf et al. 2010). After a school reform effort in New York City, newly created 
small high schools of choice increased high school graduation rates by 9 percentage points and 
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college enrollment by 8 percentage points (Bloom and Unterman 2014). Dobbie and Fryer (2015) 
showed that students who win an admissions lottery at a high-performing charter middle/high school 
in New York City were 13 percentage points more likely to graduate from high school and 17 
percentage points more likely to enroll in college immediately after graduating. Similarly, Davis and 
Heller (2019) found that winning an admissions lottery to a high-performing charter high school in 
Chicago increased college enrollment and persistence by 10 percentage points. Sass et al. (2016) 
showed that attending charter high schools in Florida increased high school graduation rates by 6 
percentage points, college attendance rates by 9 percentage points, college persistence rates by 12 
percentage points, and annual earnings from ages 23 to 25 by $2,300.° 


Another way to measure the extent to which the promotion power measures distinguish between 
schools is to examine the proportion of schools that are statistically distinguishable from average. We 
found that 53 percent of schools are statistically distinguishable from average based on promotion 
power for high school graduation, 67 percent for credential completion, 41 percent for college 
enrollment, 47 percent for college persistence, and 44 percent for earnings at age 26 (Table IV.1). 
The average standard error, a measure of the precision of the typical school’s promotion power 
estimate, ranges from 3 to 4 percentage points across the binary outcomes and is $1,456 for the 


earnings outcome. 


Table IV.1. Precision and magnitude of the promotion power measures 


Outcome 

High school Credential College College Earnings 
Measure graduation completion enrollment persistence (age 26) 
Percentage of schools 52.8 67.3 40.6 46.7 44.0 
different from 
average 
Average standard 2.7 3.0 3.1 2.6 $1,456 
error (percentage 
points or dollars) 
Standard deviation 10.4 13.4 7.2 6.2 $3,248 
(percentage points or 
dollars) 
Standard deviation 0.24 0.27 0.14 0.15 0.15 
(standard deviation 
units) 
R-squared 0.27 0.31 0.24 0.24 0.09 


The third row of Table IV.1 displays the standard deviation of the promotion power impacts, which is 
a measure of how large the difference is between the effectiveness of high- and low-performing 
schools in terms of their ability to influence student outcomes. For example, students attending a 
high school that is one standard deviation above average in the promotion power distribution for 


© Most of the studies cited here report impacts that average over multiple schools. Analyzing a single very high-performing school, such as the 
95th percentile example used in our results, would likely lead to larger estimated impacts. 
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college enrollment are 7 percentage points more likely to enroll in college than students attending an 
average school. 


Because the outcomes used for promotion power are from distributions with different student-level 
standard deviations, and because earnings are measured in units different from the other outcomes, 
the magnitude of the values in the third row of Table IV.1 are not directly comparable to each other. 
In the fourth row of Table IV.1 we report the same measures as in the third row, except we convert 
them to student-level standard deviation units so that they are all on the same scale. This conversion 
highlights that there is a larger spread among high schools’ impacts on high school graduation and 
credential completion than there is on the college and earnings outcomes, presumably because the 
college and earnings outcomes occur later in life when high schools are no longer directly 
influencing student outcomes. The standard deviations of the promotion power measures for college 
enrollment, college persistence, and earnings are comparable to the standard deviation of school test 
score value-added typically reported in the literature (Gonzalez et al. 2016; Deming 2014), while 
those for high school graduation and credential completion are larger. However, it is important to 
note that promotion power measures capture a high school’s influence on students over the course of 
four years, whereas school test score value-added typically measures a school’s impact on one year 
of achievement growth. 


The school impacts and student background characteristics included in the promotion power models 
for high school and college outcomes explain 24 to 31 percent of the variation in student outcomes, 
as measured by the R-squared values of the regressions. This is substantially less than the R-squared 
values typically found in the value-added literature, which often report values from 0.60 to 0.80 (see 
Walsh et al. 2018; Gonzalez et al. 2015). The higher R-squared values in other studies are likely in 
part due to use of a lagged version of the outcome (student test scores) as a control variable, which is 
not possible in promotion power measures (because 8th graders do not have prior measures of 


graduation, college enrollment, or earnings). 


The lower R-squared values for the promotion power models could indicate that these measures are 
less successful in removing bias than typical value-added measures, if some of the unexplained 
variation is due to unobservable student characteristics that are correlated with the schools that 
students attend. However, the R-squared values for the promotion power models for high school and 
college outcomes are also lower because of the binary nature of the outcomes. R-squared values are 
expected to be lower in regression models with binary outcomes than in similar models with 
continuous outcomes, because the model cannot possibly explain as much of the variation in the 
binary outcome (Cox and Wermuth 1992). 


The R-squared value of the promotion power model for earnings is the lowest among the outcomes 
we analyzed. The school impacts and student background characteristics explain 9 percent of the 
variation in earnings at age 26. This could be because this outcome is measured eight years after high 
school completion; because there can be substantial year-to-year fluctuations in earnings; or because 
the earnings advantage associated with higher education might not be fully evident until later than 
age 26. 
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Promotion power measures are stable over time. The performance of a high school is unlikely to 
change rapidly from one year to the next, so promotion power measures should be relatively stable. 
The year-to-year correlations of the main promotion power measures are reported in the first row of 
Table IV.2. Stability of the measures is quite high, partly by construction: one year of data overlaps 
in each two-year window used to calculate the correlation. But even when we examine results (in the 
second row of Table IV.2) for single years (which are non-overlapping), correlations remain high and 
range from 0.63 to 0.73. 


Table IV.2. Promotion power measures are stable over time 


Outcome 
High school Credential College College Earnings 
Measure graduation Completion enrollment persistence (age 26) 
Year-to-year 0.91 n/a 0.86 0.88 n/a 


correlation of main 
two-year measure 


Year-to-year 0.73 0.69 0.63 0.68 0.66 
correlation of single- 
year measure 


Notes: The high school graduation and college enrollment correlations are based on 12 years of data while the 
college persistence correlation is based on 8 years of data. Only 2 years of data were available for the 
credential completion and earnings measures, which is why it was not possible to calculate a year-to-year 
correlation of the two-year measure for those outcomes. 


n/a = not applicable. 


Schools with a positive impact on one promotion power measure are more likely to positively 
impact other student long-term outcomes, but many schools show varying effectiveness for 
different outcomes. Table IV.3 reports the correlations between promotion power measures for each 
school across the outcomes included in our analysis. In this table, we used the same two cohorts of 
students for each measure from the most recently available years, in order to isolate the relationship 
in school impacts across measures from potential changes in a school’s impact over time.’ This 
means that the correlations between high school graduation, credential completion, and college 
enrollment use the cohorts of students who graduated high school in 2016 and 2017, the correlations 
involving the earnings outcome uses the cohorts graduating in 2006 and 2007, and the correlations 
involving college persistence use the cohorts of students graduating in 2012 and 2013 (except for the 
correlation between college persistence and earnings, which uses the cohorts graduating in 2006 and 
2007). 


7 The use of the same sets of cohorts for each pair of correlations in Table IV.3 could result in correlations that are biased upwards if there are 
student-specific shocks that are positively correlated across outcomes. As a robustness check, we calculate a similar set of correlations using 
adjacent cohorts for each pair of outcomes. The results were generally lower than those reported in Table IV.3, which could in part be due to true 
changes in school effectiveness over time when adjacent cohorts are used. All the correlations that were statistically significant when overlapping 
cohorts were used remained statistically significant when adjacent cohorts were used (Appendix Table A.3). 
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Table IV.3. Promotion power impacts are positively correlated across outcomes 


Credential College College Earnings 
Outcome completion enrollment persistence (age 26) 
High school graduation 0.36 0.57 0.15+ 0.27 
Credential completion -_ 0.24 n/a n/a 
College enrollment = a 0.72 -0.07* 
College persistence a -- == -0.03* 


Notes: The correlations in this table are based on the most recent available cohorts for each pair of outcomes. 
The correlations between credential completion, college persistence, and earnings are not reported 
because we do not have data for the same set of cohorts for those outcomes. All correlations are 
significantly different from zero at the 0.05 level, except those indicated by the + symbol. 


n/a = not applicable. 


Data on credential completion were only available for the two most recent years, so we did not have 
overlapping cohorts between credential completion and the persistence and earnings measures. For 
the closest available cohort (which was four years apart) the correlation between credential 
completion and college persistence was 0.02 and not statistically significant. The correlation between 
credential completion and earnings for the closest available cohort was 0.14 and statistically 
significant, despite the fact that the cohorts were 10 years apart and the effectiveness of schools may 
have changed over that time period. 


For the majority of promotion power measures, the correlations across measures are positive and 
statistically significant. For example, schools that positively impact students’ likelihood of 
graduating from high school positively impact the credential completion, college enrollment, and 
earnings outcomes for these students as well. However, even for the correlations that are statistically 
significant, the fact that some are substantially less than 1 suggest that some schools are relatively 
better at helping students graduate from high school than they are at improving other outcomes. This 
highlights the need to include multiple promotion power measures to capture the different ways 
schools can influence student long-term outcomes. 


The low correlations between the promotion power measures for earnings at age 26 and the 
corresponding measures for college enrollment and persistence might be driven by three factors. 
First, some high schools might focus more on improving the earnings of their students than they are 
on college outcomes. For example, a recent study of high schools in Connecticut focused on career 
and technical education (CTE) found that they had positive impacts on the outcomes related to high 
school graduation and earnings at age 23 for male students, but negative impacts on the college 
enrollment outcomes of those students—presumably because they were effectively preparing their 
students for jobs that did not require college (Brunner et al. 2019). 


Second, the college earnings premium increases with age—it might be that age 26 is too early for a 
high school’s impact on college enrollment and completion to be reflected in a student’s long-term 
earnings (Card and Lemieux 2001). Third, college enrollment and persistence have much smaller 
impacts on a student’s future wages than college degree completion (Jaeger and Page 1996). As 
discussed previously, we would like to include college degree completion as an outcome for 
promotion power, but we are unable to do so because of the under-reporting of degree completion in 
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the NSC data. Appendix Section F shows differences in earnings at age 26 for students with varying 
levels of educational attainment to help explain the low correlations with the college enrollment and 
persistence measures, and why we might expect to see a substantially higher correlation if we could 
calculate a promotion power measure for college degree completion. 


Though there is a low correlation between the promotion power measures for college enrollment and 
persistence and earnings at age 26, both sets of measures might provide useful information about 
high school impacts on student outcomes. A school’s impact on enrollment and persistence might not 
translate directly to an impact on early-career wages, but the completion of some college without a 
degree has been shown to boost later-career wages after individuals have had a chance to gain work 
experience (Jaeger and Page 1996; Kane and Rouse 1995). In addition, college enrollment and 
persistence are necessary steps students must take to complete a degree, so in the absence of a 
promotion power measure for degree completion, these measures provide important indications of a 


school’s likely impact on degree completion. 


Similarly, promotion power impacts on earnings at age 26 might contain additional important 
information about a school’s impact beyond what the other measure are able to provide. If some high 
schools are especially good at preparing students for jobs that do not require college (like the CTE 
schools in Connecticut), the earnings measure should provide evidence of that. And if some high 
schools are especially good at preparing students for college graduation (which we cannot directly 
observe without an accurate measure of degree completion), the earnings measure should provide 
some evidence of that as well. 


Compared to school average outcomes, promotion power measures are much less strongly 
related to student background characteristics. One of the central goals of promotion power 
measures is to remove the bias that exists when using school average outcomes as measures of school 
effectiveness. In other words, we aim to account for and remove the effect of differences in student 
advantages and disadvantages that are not under the control of schools. We examine this by showing 
how the relationship between school average baseline characteristics and the measure of 
effectiveness change if we use school average outcomes, as opposed to promotion power, as a 
measure of effectiveness. That is, we expect that school-level baseline characteristics will drive some 
of the variation in school average outcomes, but they should not drive as much of the variation in 
promotion power.’ 


The left panels of Figures I[V.2 through IV.6 show the relationship between school poverty rates—as 
measured by the percentage of students eligible for free or reduced-price lunches (FRL)— and the 
raw outcome measures, whereas the right panels show the relationship between the school poverty 
rates and the schools’ promotion power measures. School average levels of high school graduation, 
credential completion, college enrollment, college persistence, and earnings are strongly negatively 
related to the proportion of students eligible for FRL at the school, consistent with decades of 
research. However, when looking at promotion power measures in place of the unadjusted rates, one 


8 We do not necessarily expect school average baseline characteristics and promotion power measures to be completely uncorrelated, even if the 
promotion power measures are unbiased. For example, students from low-income households (as measured by eligibility for FRL) might attend 
schools that are truly less effective than schools serving students from higher-income households. 
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finds that the negative relationship is substantially reduced for high school graduation; disappears for 
credential completion, college enrollment, and earnings; and becomes slightly positive for college 
persistence. This suggests that the promotion power measures are much more effective than raw 
outcome measures at putting all high schools on a level playing field and isolating their impacts on 
their students, regardless of the advantages or disadvantages those students might bring with them. 


Figure IV.2. Large negative relationship between high school graduation rates and school 
poverty, but no relationship for promotion power impacts 
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Notes: Each dot represents a Louisiana high school. The green line indicates the relationship between the 
proportion of students eligible for free or reduced-price lunch and each school measure. The slope of the 
regression line in the graph on the left is negative and statistically significant at the 0.05 level; the slope 
of the regression line in the graph on the right is not significantly different from zero. 


Figure IV.3. Large negative relationship between credential completion rates and school 
poverty, but no relationship for promotion power impacts 
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Notes: Each dot represents a Louisiana high school. The green line indicates the relationship between the 
proportion of students eligible for free or reduced-price lunch and each school measure. The slope of the 
regression line in the graph on the left is negative and statistically significant at the 0.05 level; the slope 
of the regression line in the graph on the right is not significantly different from zero. 
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Figure IV.4. Large negative relationship between college enrollment rates and school 
poverty, but small positive relationship for promotion power impacts 
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Notes: Each dot represents a Louisiana high school. The green line indicates the relationship between the 
proportion of students eligible for free or reduced-price lunch and each school measure. The slope of the 
regression line in the graph on the left is negative and statistically significant at the 0.05 level; the slope 
of the regression line in the graph on the right is positive and statistically significant. 


The positive relationship in the right panel of Figure IV.4 indicates that schools with higher 
proportions of students eligible for FRL have slightly higher promotion power impacts for college 
enrollment compared to schools with lower proportions of FRL-eligible students. Though this 
relationship is statistically significant, it is relatively small in magnitude. A 10 percentage point 
increase in the percentage of students eligible for FRL is associated with a 0.4 percentage point 


increase in the promotion power measure for college enrollment. 


Figure IV.5. Large negative relationship between college persistence rates and school 
poverty, but no relationship for promotion power impacts 
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Notes: Each dot represents a Louisiana high school. The green line indicates the relationship between the 
proportion of students eligible for free or reduced-price lunch and each school measure. The slope of the 
regression line in the graph on the left is negative and statistically significant at the 0.05 level; the slope 
of the regression line in the graph on the right is not significantly different from zero. 
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Figure IV.6. Large negative relationship between earnings and school poverty, but no 
relationship for promotion power impacts 


© $40,000 % $10,000 
» $35,000 © 8 
D $30,000 QE $5,000 
» $25,000 as 
ic) © 
w $20,000 3 0O—ve___—_—_——_—_———— 
2 $15,000 8 
‘Ee $10,000 Be -$5,000 
L ° 
8 $5,000 cs 

$0 S £ -$10,000 

0 0.2 0.4 0.6 0.8 1 ” ° 0 0.2 0.4 0.6 0.8 1 
Proportion of students eligible for = Proportion of students eligible for 
free or reduced-price lunch free or reduced-price lunch 


Notes: Each dot represents a Louisiana high school. The green line indicates the relationship between the 
proportion of students eligible for free or reduced-price lunch and each school measure. The slope of the 
regression line in the graph on the left is negative and statistically significant at the 0.05 level; the slope 
of the regression line in the graph on the right is not significantly different from zero. 
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A. Additional details about outcome measures 


High school graduation. We measured graduation as a traditional high school diploma from a public 
Louisiana high school. Similar to the definition of graduation used for LDOE’s school accountability 
system, students who drop out of high school to complete a General Education Degree, or transfer 
out of Louisiana public schools to a private school or a school in another state, are counted as not 
graduating from high school for the purposes of the promotion power analysis. 


The high school graduation rate in this report differs from the one reported by LDOE because two 
cohorts of students are used for our promotion power model, whereas LDOE reports single-year 
graduation rates. 


Credential completion. Though students might complete the courses necessary to earn a credential 
before graduating high school, this outcome is measured at the time of high school graduation such 
that all students who do not graduate from high school on time are also not counted as completing a 


credential. 


Similar to the high school graduation outcome, the credential completion rate in this report differs 
from the one reported by LDOE because two cohorts of students are used for our promotion power 
model, whereas LDOE reports single-year credential completion rates. 


College enrollment. LDOE obtains college enrollment data from the NSC by sending a list of 
student names and birth dates to the NSC to be matched with its college data obtained from 
participating institutions. Nearly all public and private nonprofit colleges, and a majority of for-profit 
colleges, report data to the NSC, such that the NSC data cover 97 percent of college students (NSC 
2019). For this outcome, we included enrollment that occurred from August to December of the 
academic year following the spring the student was expected to graduate from high school. 


We also explored using a college enrollment outcome based on whether the student attended college 
in either of the first two academic years following the spring the student was expected to graduate 
from high school. We estimated the correlation between school promotion power estimates on college 
enrollment using these two versions of the outcome, and found a correlation of 0.97. Because the two 
measures produce very similar results, we recommend using the measure based on only one academic 
year because the promotion power results would be available one year closer to the time when 
students were in high school. 


LDOE only sends data from students who graduate from high school to the NSC for matching 
purposes, meaning that students who are not counted as graduating from high school in the promotion 
power models are assumed not to enroll in college within one year of their expected high school 
graduation date. 


A small number of students were recorded as graduating from high school but their data were not 
sent to the NSC for matching purposes, and thus could not have been recorded as attending college. 
These students were dropped from all models. 
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The college enrollment rate in this report differs from the one reported by LDOE because the college 
enrollment rate we report is a percentage of all students who enrolled in Louisiana high schools in 
9th grade, whereas the college enrollment rate reported by LDOE is a percentage of students who 
graduated from high school. 


College persistence. LDOE and other researchers (Dynarski et al. 2015) have found that the NSC 
measure of college graduation is incomplete. Because the NSC’s main objective is to determine 
whether a given student is enrolled in college at a given point in time, the data are inconsistent in 
capturing whether a student earned a degree. As a result, while students who were recorded as 
graduating in the NSC data can be assumed to have actually graduated, a student who actually 
graduated may or may not be recorded as having done so in the data. We therefore constructed the 
outcome measure for multi-year college persistence to capture college completion as closely as the 
data would allow. 


Because students who were recorded as graduating from a four-year college can be considered true 
graduates, we examined the college-going patterns of these students to develop our measure of 
persistence that would approximate college degree completion. Generally, these students completed 
at least four years of college in total, and at least two of those years were at a four-year college. We 
ultimately built the measure of multi-year college persistence that counts students as persisting if 
they attended college for at least four years, at least two of which were at a four-year college. The 
small percentage of students who we did not observe persisting for at least four years but who 
received a bachelor’s degree in the NSC data also counted toward this persistence measure. 


National data indicate that earning a bachelor’s degree takes more than four years for over half of 
college students (Woo et al. 2012), which indicates that we should wait more than four years to 
evaluate whether students have completed a bachelor’s degree. On the other hand, if we examined 
our measure too many years after a given cohort was expected to graduate high school, it would 
increase the time between when promotion power is measured and when that cohort attended high 
school. To balance the priorities of accurately approximating college completion with limiting the 
time between when the outcome is measured and when students attended high school, we measured 
multi-year college persistence using the five years after a student’s cohort was expected to graduate 
high school. 


Earnings. Data on all 9th- through 12th-grade students enrolled in Louisiana public high schools 
during the years 2005—06 through 2014-15 were sent to the LWC to be matched to unemployment 
insurance records for calendar years 2013-2015. Matching was done based on student Social Security 
number (SSN). LDOE stopped collecting data on student SSNs in recent years, and is currently in 
discussion with the LWC about obtaining earnings data in subsequent years using alternative 
matching methods. Data on all students in our analysis sample were sent to the LWC for matching 
with two exceptions: (1) those with missing or inaccurate SSNs were not sent to the LWC, and 

(2) LDOE recently restructured its student ID system, which resulted in some students in historical 
years not being assigned student IDs. In total, less than 2 percent of students in our analysis sample 
were excluded from the promotion power models for earnings because their data were not sent to the 
LWC for matching. 
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B. Additional details about background characteristics 


With the exception of whether students transferred between high schools and the over-age for grade 
variable, all data are measured during the year the student was in 8th grade. 


8th-grade test scores. To account for changes in score scaling and content across years, we 
standardized the LEAP test scores in ELA, math, science, and social studies separately by subject and 
year to have a mean of 0 and standard deviation of 1 before using the scores to calculate promotion 
power. We used the most recent test score for students who repeated 8th grade. 


8th-grade attendance and suspensions. The models use a measure of 8th-grade attendance based on 
the annualized number of absences a student had in 8th grade. That is, if a student attended 8th grade 
for only half the year (90 school days) and was absent for 10 of those days, his or her annualized 
absences would be set to 20 (based on a 180-day school year). We also top-coded annualized 
absences at 60 to reduce the influence of outliers. 


Suspensions are based on the total number of in-school and out-of-school suspensions students 
received in 8th grade. We annualized suspensions in a similar way as absences and top-coded the 
annualized number of suspensions at 18 to reduce the influence of outliers. 


Transfer high schools. School transfers can be disruptive to students’ high school academic 
achievement and could lead to worse long-term outcomes. Ideally, if students transfer out of schools 
because the schools are negatively impacting their long-term outcomes, we would attribute this 
transfer behavior to the low-performing high schools themselves and not include an indicator for it. 
However, because the majority of student transfers likely occur for reasons outside the control of 
schools (Rumberger 2015), we chose to include a transfer indicator in the promotion power models. 
Including this indicator prevents schools that receive large numbers of transfer students from 


experiencing negative impacts in their promotion power scores as a result. 


Free or reduced-price lunches. For all the promotion power models described in this report, we 
used indicators for FRL eligibility as a measure of poverty level. Starting in 2014, Louisiana 
switched from using FRL status as a measure of student poverty to an indicator of economic 
disadvantage. Students are considered economically disadvantaged if they are eligible for the 
Supplemental Nutrition Assistance Program, Temporary Assistance for Needy Families, or Medicaid; 
receive free or reduced-price lunch; or are categorized as limited English proficient, homeless, 
migrant, in foster care, or incarcerated. In future years, promotion power models will use the 


economic disadvantage indicator in place of FRL status. 


Disability variables. The promotion power models for cohorts graduating from 2010 onward include 
indicators for six disability categories: Emotional Disturbance, Learning Disability, Intellectual 
Disability, Other Health Impairment, Speech Impairment, and other disability. These variables were 
not available for prior cohorts, so we instead used an indicator for whether the student received 
special education services (and an indicator for the students in these cohorts for whom special 


education information missing). 
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Table A.1 displays averages and across-school standard deviations for each background characteristic 
used as a control variable in the promotion power models. 


Table A.1. Summary statistics of background characteristics 


Across-school standard 


Variable Average deviation 
LEAP ELA standardized score 0.08 0.57 
LEAP math standardized score 0.07 0.50 
LEAP science standardized score 0.07 0.54 
LEAP social studies standardized score 0.07 0.51 
Missing LEAP ELA score 0.3% 2.4 pp 
Missing LEAP math score 0.5% 2.5 pp 
Missing LEAP science score 0.5% 2.3 pp 
Missing LEAP social studies score 0.7% 4.5 pp 
8th-grade absences: Annualized 8.8 days 4.9 days 
Over-age for grade 8.1% 16.5 pp 
Transfer high schools 21.3% 32.8 pp 
Free lunch receipt 57.0% 23.4 pp 
Reduced-price lunch receipt 7.9% 9.4 pp 
Limited English proficiency 1.2% 1.9 pp 
8th-grade suspensions: Annualized 0.6 days 1.0 days 
Disability: Emotional Disturbance 0.2% 5.5 pp 
Disability: Learning Disability 2.8% 9.6 pp 
Disability: Intellectual Disability 0.1% 0.8 pp 
Disability: Other Health Impairment 1.3% 6.1 pp 
Disability: Speech Impairment 1.0% 1.6 pp 
Other disability 0.6% 7.9 pp 
Gifted status 3.7% 5.3 pp 
Attend 8th grade fewer than 45 days 0.3% 1.4 pp 


Notes: Summary statistics of background characteristics are based on data from the two cohorts of students 
entering the 2015-16 and 2016-17 high school graduation promotion power models. These students 
were in 9th grade during the 2012-13 and 2013-14 school years. 


pp = percentage points. 


C. Additional model details 


Shrinkage procedure. To accommodate that we are combining estimates across years, we first 
shrunk each cohort-specific estimate for each school, then combined the two shrunk estimates across 
two cohorts. We adopted this approach, rather than first combining and then shrinking, to allow for 
the reporting of single-year measures that can be averaged together to produce the main two-year 
measures. However, without further adjustment this would result in estimates that are moved toward 
the mean by too much: the precision for the single-cohort estimates, which informs how much 
shrinkage should be induced, is much less than the precision of the combined estimates that use two 
cohorts. Thus, we applied an adjustment by scaling the estimated variance of the cohort-specific 
estimates by 1/2, applying shrinkage based on this reduced variance, then scaling the variance of the 
shrunk estimate by 2. We then combined these adjusted estimates and variances as described above. 
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Setting the sum of the weights equal to 1 for all students. We set the total weight, across all 
observations for a student, to 1. This means that a student who attended all four years at a high 
school contributes the same weight to that high school as a student who attended in 9th grade and 
then left Louisiana public schools or dropped out. We chose to do this because we cannot distinguish 
students who drop out of high school from those who leave state public schools. Because dropping 
out is related to the outcome of graduating in four years (as well as later outcomes), reducing the 
contributions of students based, in part, on their outcome would introduce bias into the promotion 


power measures. 


Addressing out-of-range expected outcomes. For the credential completion and college outcomes, 
there are some cases where zero percent of students at a school achieve a given outcome. Because we 
are using a linear probability model, it is possible for the regression model to produce an expected 
outcome rate of less than zero percent for these schools. We removed these schools from the results 
distribution, because the model is not able to provide useful information about a school’s impact on 
the outcomes of its students in such cases. Less than 1 percent of schools in the credential completion 
model and college enrollment models, and 7 percent of schools in the college persistence model were 
removed as a result of this restriction. 


D. Robustness checks 


We made a number of data and modeling decisions when developing the promotion power measures. 
To check the sensitivity of the results to these decisions, we estimated a number of alternative 
models as robustness checks.° We measured the sensitivity of the results by correlating estimates of 
school promotion power from the main model with those from each alternative model. A high 
correlation indicates that we would obtain similar estimates if we used the alternative model, so the 
main model is robust to the data or modeling decision in question; a lower correlation suggests that 
the decision has a more substantive influence on the results. The results of the robustness checks are 
displayed in Appendix Table A.2. The correlations range from 0.85 to >0.99, which is consistent with 
other studies comparing the sensitivity of value-added model results to different modeling decisions 
(Guarino et al. 2015; Johnson et al. 2015; Goldhaber et al. 2013). 


Logistic regression. Our main model uses a linear probability model, as discussed in Chapter III, 
Section 4, though it is also common to use logistic regression when the outcome variable is binary. 
For this robustness check, we estimated a logistic regression. The logistic regression automatically 
drops high schools that have no variation in the outcome from the estimation sample, along with all 
of their students. To measure the extent of differences for a consistent set of schools, for the purpose 
of this robustness check we estimated a linear probability model using only those schools and 
students that were kept in the logistic regression. The correlations range from 0.85 to 0.98. 


Excluding students with missing test scores and background characteristics. In our main model, 
we included students who are missing one or more LEAP test scores (math, ELA, social studies, and 


° Fewer robustness checks are performed for the earnings results because they are substantially more time consuming due to the security 
measures governing access to those data. 


A-6 


PROMOTION POWER TECHNICAL REPORT -APPENDIX MATHEMATICA 


science) as long as they are not missing all scores and include students who have unreliable data on 
8th-grade absence rates and suspensions due to being enrolled in Louisiana schools for fewer than 45 
school days. In this robustness check, we instead excluded these students from the sample. 
Correlations exceed 0.99 in all cases. 


Cubic specification pre-test scores. In our main model, we assumed a linear relationship between 
each of the pre-test scores and the outcome. We did not use a polynomial specification both for 
simplicity and because research has shown that polynomial specifications of prior test scores can 
exacerbate problems related to test score measurement error (Lockwood and McCaffrey 2014). 
However, polynomial functions of prior test scores are frequently used in the literature (for example, 
Chetty et al. 2014a; Mansfield 2015; Jackson 2014), and it is possible the true relationship between 
long-term outcomes and 8th-grade test scores is non-linear. Therefore, as a robustness check, we 
allowed the relationship for each pre-test score to be cubic. 


The correlations between the main model and the specification with cubic prior test scores are all at 
least 0.99. 


Including gender and race/ethnicity variables. In our main model, we did not use indicators for 
students’ gender or race/ethnicity for consistency with the school and teacher value-added models 
LDOE uses. However, many value-added models of teacher and school effectiveness include these 
characteristics (for example, Gonzalez et al. 2016; Chetty et al. 2014a; Walsh et al. 2014). In these 
robustness checks, we added indicators for race/ethnicity and gender, along with an indicator for 
students who were missing data on gender. For the model that includes race/ethnicity variables, the 
correlations ranged from 0.91 to >0.99. They were lowest for the college enrollment and persistence 
outcomes, indicating a stronger relationship between race/ethnicity and college outcomes than for 
high school graduation, after accounting for the other control variables included in the models. For 
the model that includes gender, all of the correlations were at least 0.99. 


Table A.2. Correlations between results from robustness check models and the main model 


Polynomial No missing 
Logistic 8th-grade baseline Add race Add gender 
Outcome regression test scores covariates covariates covariates 
High school graduation 0.98 >0.99 >0.99 0.98 >0.99 
Credential completion 0.92 >0.99 >0.99 >0.99 >0.99 
College enrollment 0.92 >0.99 >0.99 0.91 >0.99 
College persistence 0.85 0.99 >0.99 0.93 0.99 


Alternate promotion power models for earnings. In the main model, we exclude students who 
attended an out-of-state college and did not appear in the earnings data. As robustness checks, we 
also estimated models in which we (1) included all students whose data were sent to the LWC for 
matching, and (2) excluded all students who did not appear in the earnings data. When we included 
all students, the correlation of promotion power impacts with those from the main model was 0.99. 
When we excluded all students with no earnings data, the correlation was 0.93. 
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Correlations across outcomes using adjacent cohorts. The results reported in Table IV.3 use the 
same sets of cohorts for each pair of correlations, which could result in correlations that are biased 
upwards if there are student-specific shocks that are positively correlated across outcomes. As a 
robustness check, we calculated a similar set of correlations using adjacent cohorts for each pair of 
outcomes. For example, for the correlation between promotion power measures for high school 
graduation and credential completion, we used the high school graduation measure based on the 
cohorts that graduated in 2016 and 2017 and the credential completion measure based on the cohorts 
that graduated in 2014 and 2015. 


Using adjacent cohorts, rather than the same cohorts, generated correlations that are generally lower, 
as expected (Table A.3). The lower correlations are likely caused by two factors: (1) using adjacent 
cohorts avoids contamination from student-specific shocks that are positively correlated across 
outcomes, and (2) high school’s effectiveness changes over time (as demonstrated in the second row 
of Table IV.2), and the adjacent cohorts are as far as three years apart. 


Table A.3. The correlations across promotion power measures are similar when adjacent 
cohorts are used 


Credential College College Earnings 
Outcome completion enrollment persistence (age 26) 
High school graduation 0.26 0.44 0.19 0.26 
Credential completion -- 0.11 n/a n/a 
College enrollment -- == 0.60 -0.05* 
College persistence -- -- -- -0.05* 


Notes: The correlations in this table are based on the most recent available adjacent cohorts for each pair of 
outcomes. The correlations between credential completion, college persistence, and earnings are not 
reported because we do not have data for adjacent cohorts for those outcomes. All correlations are 
significantly different from zero except those indicated by the + symbol. 


n/a = not applicable. 
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E. Precision increases as cohorts are added 


Chapter II, Section B.4, discusses how the precision of school estimates changes as additional 
cohorts are included in the measure. Using additional cohorts of students increases precision, and it 
adds a year between the time the measure can be estimated and when the students included in the 
model were in high school. In addition, the benefit in precision decreases as cohorts are added: the 
gain in precision is greater when going from one cohort to two cohorts than when increasing from 
two cohorts to three. Our main model balances the advantages and disadvantages of additional 
cohorts by using two cohorts for each measure. Table A.4 shows how the proportion of schools 
significantly different from average increases, and the average standard error generally decreases, as 
we add cohorts. 


Table A.4. The precision of the promotion power measures increases with the number of 
cohorts used in the calculations 


High school Credential 

graduation completion College enrollment College persistence 
Number of 
cohorts 1 2 3 1 2 3 1 2 3 1 2 3 
Proportion 41.4 52.8 57.0 60.1 67.3 n/a 27.4 40.6 44.4 33.0 46.7 47.5 
statistically 
different from 
average (%) 
Average 3.5 2.7 2.5 4.1 3.0 n/a 3.8 3.1 2.7 3.2 2.6 2.7 
standard error 
(percentage 
points) 


n/a = not applicable. 


F. Earnings premiums by educational attainment 


As discussed in Chapter IV and illustrated in Table IV.3, we found no correlation between the 
promotion power measures for college enrollment and persistence and the promotion power measure 
for earnings. This finding is unexpected: if a school is better than average at getting its students to 
attend and persist in college, and those outcomes lead to higher earnings, we would expect these 
same high schools to be better than average at promoting earnings. To explore this finding in more 
detail, we examined differences in earnings by levels of secondary and postsecondary educational 


attainment, often referred to as earnings premiums. !° 


We start by presenting two figures that show the earnings data behave as one would expect, in that 
average earnings increase with education level and the earnings premium associated with college 
enrollment increases with age. We then show in a third figure that, after accounting for the variables 
included in the promotion power models, the earnings premium at age 26 associated with college 


'0 For analyzing these premiums, we examined students who graduated from a four-year college, according to the NSC, within six years of when 
they should have graduated from high school. However, the NSC does not capture college graduation data for all students, so we could not use 
this outcome as a promotion power measure, as it would have underestimated promotion power for high schools that might have many graduates 
attending colleges that did not report graduation data to the NSC (Chapter II, Section A.4, provides additional details). 
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enrollment and persistence are lower than the premium associated with high school graduation, which 
helps explain the low correlations between those promotion power measures and the promotion 


power measures for earnings. 


First, earnings at age 26 increase as educational attainment increases, and the biggest difference is 
between students who did not graduate high school and those who have (Figure A.1). 


Figure A.1. Average earnings at age 26 increase with educational attainment 
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Notes: For each category other than students who did not graduate from high school in four years, all students 
who reached at least that level of educational attainment are included. For example, the high school 
graduate category includes students who went on to enroll in college, persist in college, and so on. 


Second, the earnings premiums in Figure A.1 are likely to grow as the students age. As an example, 
we show that the premium for enrolling in college increases from age 23 to age 26, suggesting it is 
likely to continue to grow with age (Figure A.2). 


Figure A.2. The earnings premium for college enrollment increases with age 
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Notes: The premium for college enrollment is the difference in earnings between all those students who enrolled 
in college in the fall of the year they should have graduated from high school (which includes those who 
went on to persist in college and/or graduate) and all those students who did not enroll (which includes 
students who graduated from high school and those who did not). 
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Finally, the sizes of adjusted earnings premiums for different categories help explain the pattern of 
correlations across promotion power measures discussed in Chapter IV. To more closely match the 
promotion power measures, we calculated adjusted premiums associated with each secondary and 
postsecondary educational attainment outcome. To calculate the adjusted premium, we regressed 
earnings at age 26 on an indicator for achieving the specified educational attainment and included all 
of the student characteristics in the promotion power models (8th-grade test scores, student absences, 
and so on). We ran this regression separately for each educational attainment outcome. The 
coefficient on the educational attainment indicator is the adjusted premium. 


The adjusted premiums are highest for high school graduation and four-year college degree 
attainment and are lower for college enrollment and persistence (Figure A.3). This finding is in line 
with the patterns we observe in the correlations in Table IV.3, where the promotion power measure 
for high school graduation has a significant positive correlation with the promotion power measure 
for earnings, while the promotion power measures for college enrollment and persistence have a low 
correlation with the promotion power measure for earnings. The high adjusted premium for four-year 
college degree completion suggests that if Louisiana were able to capture this outcome for all 
students and we could calculate a promotion power measure for degree completion, the resulting 
measure would also be significantly positively correlated with earnings at age 26. 


Figure A.3. Adjusted premiums are highest for high school graduation and four-year college 
degree completion 
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Notes: The premium for each category is the difference in earnings between all those students who achieved at 
least that level of graduation and all those students who did not, after controlling for background 
characteristics. For example, the college persistence premium is the difference in earnings between all 
students who persisted in college (including those who went on to graduate), and those who did not 
(including those who did not graduate high school, graduated high school, or enrolled in college). 
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G. Coefficients on background characteristic variables 


Table A.5 lists the coefficients, standard errors, and significance levels for all of the test scores and 


student background characteristics used in the promotion power models, separately for each outcome. 


Interpreting multivariate regression results. The coefficients displayed below might not reflect the 
relationship we would have observed if the other characteristics had not been accounted for in the 
model. Multiple regression coefficients can produce counterintuitive relationships between characteristics 
and the outcome in cases where the contribution of one characteristic is largely accounted for by a 


different characteristic in the model (Isenberg and Walsh 2014). For example, the coefficients on limited 
English proficiency status would likely be negative and larger in magnitude if the model did not also 
account for students’ 8th-grade test scores, because students with limited English proficiency tend to 
have lower 8th-grade test scores. 


Table A.5. Regression coefficients, standard errors, and significance levels for each model, 


by outcome 


Background characteristic 


LEAP ELA score 

LEAP math score 

LEAP science score 

LEAP social studies score 
Missing LEAP ELA score 

Missing LEAP math score 
Missing LEAP science score 
Missing LEAP social studies score 
8th-grade absences: Annualized 
Over-age for grade 

Transfer high schools 

Free lunch receipt 
Reduced-price lunch receipt 
Limited English proficiency 


8th-grade suspensions: Annualized 


High school 


graduation 


0.041*** 
(0.003) 
0.036*** 
(0.003) 
-0.011*** 
(0.003) 
0.026*** 
(0.003) 
-0.088 
(0.045) 
-0.064* 
(0.031) 
0.084* 
(0.033) 
-0.076** 
(0.027) 
-0.010*** 
(0.000) 
-0.243*** 
(0.009) 
-0.108*** 
(0.006) 
-0.032*** 
(0.004) 
-0.011 
(0.007) 
0.006 
(0.017) 
-0.029*** 
(0.001) 


Credential 
completion 
0.078*** 
(0.004) 
0.070*** 
(0.003) 
0.024*** 
(0.004) 
0.039*** 
(0.004) 
-0.187*** 
(0.038) 
-0.004 
(0.026) 
0.082** 
(0.030) 
0.054* 
(0.022) 
-0.005*** 
(0.000) 
-0.097*** 
(0.007) 
-0.100*** 
(0.005) 
-0.080*** 
(0.005) 
-0.045*** 
(0.008) 
0.133*** 
(0.018) 
-0.018*** 
(0.001) 


College 
enrollment 
0.089*** 
(0.004) 
0.063*** 
(0.003) 
-0.016*** 
(0.004) 
0.026*** 
(0.004) 
-0.219*** 
(0.033) 
-0.011 
(0.028) 
0.085** 
(0.028) 
0.058* 
(0.023) 
-0.007*** 
(0.000) 
-0.147*** 
(0.007) 
-0.119*** 
(0.006) 
-0.097*** 
(0.005) 
-0.071*** 
(0.008) 
-0.009 
(0.018) 
-0.018*** 
(0.001) 


College 
persistence 
0.068*** 
(0.003) 
0.072*** 
(0.003) 
-0.006 
(0.004) 
0.037*** 
(0.003) 
-0.048 
(0.041) 
0.019 
(0.036) 
0.067** 
(0.024) 
0.042* 
(0.017) 
-0.004*** 
(0.000) 
-0.040*** 
(0.004) 
-0.075*** 
(0.004) 
-0.080*** 
(0.005) 
-0.058*** 
(0.007) 
0.037* 
(0.015) 
-0.010*** 
(0.001) 


Earnings at 
age 26 
-1428.29*** 

(213.07) 
2924.21*** 
(200.61) 
988.11*** 
(221.61) 
683.82** 
(216.26) 
-221.69 
(916.89) 
-4293.85*** 
(1109.47) 
292.85 
(1213.59) 
40.00 
(1137.88) 
-172.51*** 
(9.83) 
-668.12* 
(333.80) 
-1893.80*** 
(238.51) 
-3612.66*** 
(239.15) 
-2010.43*** 
(385.96) 
-3952.53** 
(1255.30) 
-490.23*** 
(68.30) 
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High school Credential College College Earnings at 
Background characteristic graduation completion enrollment persistence age 26 
Gifted status -0.028*** 0.057*** 0.035*** 0.069*** -2854.38*** 
(0.007) (0.010) (0.011) (0.012) (538.65) 
Attend 8th grade fewer than 45 days -0.311*** -0.198*** -0.204*** -0.077*** -4864.67* 
(0.043) (0.039) (0.040) (0.022) (1840.43) 
Disability: Emotional Disturbance -0.093* -0.051 -0.067 -0.013 
(0.043) (0.035) (0.037) (0.018) 
Disability: Learning Disability -0.004 -0.022 -0.050*** 0.008 
(0.013) (0.012) (0.012) (0.006) 
Disability: Intellectual Disability -0.131* 0.094 -0.017 0.129*** 
(0.066) (0.056) (0.056) (0.035) 
Disability: Other Health Impairment -0.028 -0.045** -0.084*** -0.047*** 
(0.019) (0.017) (0.017) (0.010) 
Disability: Speech Impairment 0.047** 0.016 0.030 -0.016 
(0.018) (0.021) (0.022) (0.015) 
Other disability 0.064* -0.018 0.043 0.063* 
(0.025) (0.028) (0.028) (0.025) 
Special education -1001.97* 
(403.40) 
Special education missing -2448.56* 
(945.36) 
Number of students 44,760 44,760 44,760 42,681 41,137 
R-squared 0.27 0.30 0.24 0.24 0.09 


Note: Robust standard errors are in parentheses. 


#4 PD << 0.001; ** p < 0.01; * p <0.05. 


PROMOTION POWER TECHNICAL REPORT -APPENDIX MATHEMATICA 


The coefficients, standard errors, and significance levels for the race/ethnicity and gender 
coefficients, estimated in separate robustness check models, are shown in Table A.6. 


Table A.6. Regression coefficients, standard errors, and significance levels for 
race/ethnicity and gender for each model, by outcome 


High school Credential College College 
Variable graduation completion enrollment persistence 
Hispanic -0.002 0.012 -0.032** 0.017 
(0.011) (0.012) (0.012) (0.014) 
American Indian/Alaska Native 0.020 -0.014 -0.023 0.014 
(0.015) (0.019) (0.019) (0.021) 
Asian/Pacific Islander 0.019 0.051*** 0.077*** 0.155*** 
(0.013) (0.015) (0.017) (0.019) 
Black 0.092*** 0.002 0.144*** 0.108*** 
(0.005) (0.006) (0.006) (0.005) 
Female 0.057*** 0.089*** 0.106*** 0.107*** 
(0.004) (0.004) (0.004) (0.004) 
Missing gender -0.181** -0.058 0.059 -0.043 
(0.066) (0.049) (0.067) (0.066) 
Number of students 44,760 44,760 44,760 42,681 


Notes: The coefficients in this table were estimated in two separate models for each outcome. In one model, 
only the race/ethnicity variables were added; in another model, only the indicator for being female and 
the indicator for having missing data on gender were added. Robust standard errors are in parentheses. 


#4 PD << 0.001; ** p < 0.01; * p <0.05. 


